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Executive Summary 


The significant growth of charter schools in the United States has brought praise for the excellent results achieved by 
some schools as well as criticism that charter schools may not be serving the most disadvantaged students. 

Critics of charter schools, in New York City and elsewhere, commonly assert that charters' (often) strong academic 
performance derives primarily from the type of student educated, rather than the quality of schooling provided. In 
particular, many charter school opponents argue that charters systematically "push out" low-performing, or otherwise 
difficult-to-educate, students in order to boost aggregate test scores. 

This paper uses longitudinal NYC student-level enrollment data to assess such claims. Key findings include: 

• Low-performing students are more mobile, regardless of where they are enrolled: in NYC charters as well 
as traditional public schools, low-performing students are more likely to change schools than their higher- 
performing peers. 

• Low-performing students are not more likely to exit NYC charters than traditional public schools. 

• To the extent that higher attrition rates for low-performing NYC students offer cause for concern, they are no 
less a problem for the city's traditional public schools than they are for its charters. 
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Pushed Out? 

Low-Performing Students 
and New York City 
Charter Schools 

Marcus A. Winters INTRODUCTION 

cc [Charters are] free to push out low-scoring students and send 
them back to the local public school,” writes Diane Ravitch, 
a prominent charter critic. “This improves their results, but it 
leaves regular public schools with disproportionate numbers 
of the most challenging students .” 1 Carmen Farina, New York City 
schools chancellor, adds: “There shouldn’t be a whole movement out 
of charters the month before the test .” 2 

Given the frequency and assuredness of such criticism, one would be 
forgiven for assuming that it was backed by evidence more substantial 
than mere anecdotes. To date, however, no empirical evidence has 
been mustered to suggest that: (i) low-performing students are more 
likely to exit charters than traditional public schools; and (ii) charters 
systematically remove, or pressure out, low-performing students. 

Indeed, analysis of NYC data by SchoolBook (an education website) 
and the Independent Budget Office (NYC’s version of the Congres- 
sional Budget Office) finds that, on average, charters display lower 
overall attrition rates than do traditional public schools . 3 The IBO 
also finds that student attrition — when compared by gender, race/ 
ethnicity, poverty, and English learner status — in charters is lower 
than in traditional public schools . 4 Likewise, previous data analysis 
of NYC charters and traditional public schools (conducted by this 
author) found that students with disabilities 5 and students learning 
English 6 are less likely to exit charters than traditional public schools. 

Nevertheless, current research offers little evidence comparing attrition 
patterns in charter and traditional public schools that is related to prior 
student performance on standardized tests. To the best of this author’s 
knowledge, a recent paper by Vanderbilt University’s Ron Zimmer and 
Indiana University’s Cassandra Guarino provides the only such analysis 
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directly examining whether low-performing students 
are more likely to exit charters than traditional public 
schools within a large school district. 7 

Analyzing student-level data from a large, anony- 
mous midwestern school district, Zimmer and Gua- 
rino found that low-performing students exit char- 
ters and traditional public schools at equal rates. 
Though their results are convincing for the school 
district analyzed, Zimmer and Guarino urge simi- 
lar research in other cities — a reasonable suggestion, 
given that the operation and effectiveness of char- 
ters vary dramatically across school systems. 

This paper uses NYC student-level data, over six 
years, to study whether low-performing students 
are more likely to exit charters than traditional 
public schools. Low-performing students, the pa- 
per finds, are more likely to exit NYC charters than 
are higher-performing students. That pattern, the 
paper also finds, exists equally within the tradi- 
tional public school sector. In short, the author 
finds no empirical evidence that low-performing 
students are more likely to exit charter schools 
than traditional public schools. 

Though this paper does not attempt to explain why 
students exit certain schools, its findings are incon- 
sistent with the argument that NYC charters sys- 
tematically push out low-performing students. 

1. DATA 

This paper utilizes a student-level, longitudinal data 
set — made available by NYC’s Department of Edu- 
cation (DOE) — covering all students enrolled from 
2006-07 through 201 1-12 in NYC charter and tra- 
ditional public schools. Unique (albeit anonymous) 
student identifiers allow individual students to be 
tracked over time. Similarly, unique school mark- 
ers identify whether students are enrolled in charter 
or traditional public schools. Data include students’ 
demographic information and relevant test scores 
on state-mandated math and reading assessments, 
administered in grades three through eight. 

The ability to follow individual students over time 
allows the author to observe whether students exit 


their schools, as measured by: (i) their enrollment 
in a different school from the previous year; or (ii) 
their complete absence in the data set one year after 
being recorded as enrolled the previous year (i.e., 
students leave NYC’s public school system entirely). 

This paper evaluates student exits according to pre- 
vious test-score performance. State-mandated test- 
ing in New York extends from grades three through 
eight. The paper’s analysis thus begins with students 
moving into fourth grade (students entering third 
grade do not have a mandatory test score from the 
previous year, with the exception of students who 
are required to repeat third grade). Similarly, because 
mandatory testing ends in eighth grade, this paper is 
unable to analyze student exits in high school. 

The author also excludes students progressing into 
sixth grade, a “gateway” grade in which the major- 
ity of NYC traditional public school students make 
a structural move into middle school: according to 
the author’s calculations using the DOE data set, 78 
percent of such students entering sixth grade enter a 
new school, far higher than any other grade level con- 
sidered. Because this move is structural, and thus un- 
likely to be systematically influenced by achievement, 
students transitioning from fifth to sixth grade are ex- 
cluded from the analysis. Further, the author cannot 
evaluate student exits in 2006-07, the DOE data set’s 
first year, because the school in which students were 
enrolled in the previous year is not included. 

This paper’s final sample, therefore, on which all 
subsequent results are based, includes students in 
NYC charter and traditional public schools entering 
grades four, five, seven, and eight for school years 
2007-08 through 201 1-12. 

II. DEFINING LOW-PERFORMING 
STUDENTS 

One difficulty with analyzing attrition patterns of 
low-performing students is that there are many ways 
to define “low performance.” This paper selects two 
methods. 

The first classifies students as low-performing if 
their test scores fall below the NYC average for stu- 
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dents in the same grade and year. This method es- 
sentially holds all NYC public school students to 
the same academic standard. 

The second method classifies students as low-per- 
forming if their test scores fall below the average for 
students in their particular school, grade, and year. 
Using this method, students who score poorly rela- 
tive to other NYC students overall might nonethe- 
less be classified as high-performing if classmates at 
their particular school score even worse. 

To focus on NYC’s lowest-performing students, this 
paper alters the definition of “low-performing” to 
consider attrition patterns for students with test 
scores below the 25th percentile — within NYC and 
particular schools. 

III. DESCRIPTIVE ANALYSES 

The author first considers basic descriptions of 
the probability that students exit their schools, 
by performance and sector. Because such analyses 
include no statistical controls, they should be in- 


terpreted carefully. Nevertheless, they offer useful 
basic information and help set the stage for later 
statistical analyses. 

Figure 1 compares the proportion of high- and low- 
performing students who exited their schools the 
following year. The solid black bars represent stu- 
dents in traditional public schools; the striped bars 
represent students in charters. 

Figure 1 shows that low-performing students are 
significantly more likely to exit charters than are 
charter students scoring above the performance 
threshold — a result seemingly consistent with ac- 
cusations that charters systematically remove low- 
performing students. 

Yet Figure 1 also makes clear that attrition differ- 
ences among low- and high-performing charter stu- 
dents only tell part of the story. Indeed, low-per- 
forming students in traditional public schools also 
demonstrate similar attrition differences relative to 
their high-performing peers. In other words, low- 
performing students are more likely to exit their 


Figure I. Proportion of Students Who Exited School 


Below NYC Mean, Math 
Above NYC Mean, Math 

Below NYC Mean, ELA* 
Above NYC Mean, ELA 

Below School Mean, Math 
Above School Mean, Math 

Below School Mean, ELA 
Above School Mean, ELA 

Below School 25th Percentile, Math 
Above School 25th Percentile, Math 

Below School 25th Percentile, ELA 
Above School 25th Percentile, ELA 



■ Traditional Public School a Charter School 


*New York State's English Language Arts exam 

Source: Author's calculations, using data set provided by NYC Department of Education 
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schools, regardless of whether they are enrolled in 
charters or traditional public schools. 

Rather than being objects of discrimination, low- 
performing students, such descriptive analysis sug- 
gests, are simply more mobile, on average, than 
high-performing students. To confirm this finding, 
further analysis, using statistical controls and allow- 
ing for significance testing, is required. 

IV. EMPIRICAL METHOD 

The author next considers an empirical model that 
controls for student characteristics and allows for 
significance testing. The basic approach estimates 
an OLS regression (resulting in a linear probabil- 
ity model) predicting the relationship between the 
likelihood that students exit their schools, based on 
whether they are low-performers and whether they 
attend charters or traditional public schools, con- 
trolling for grade, year, and student characteristics. 
Formally, the estimation model, Equation 1, takes 
the form: 

exit. =B n + B, charter. + BX. + Bdelowtest. + 

igst + 1 1 0 1 1 igst 1 T igst 1 3 igst 

B .(charter* belowtest) . + A + 8 + £ ) 

' 4 v 7 igst t g igstr 

Where exit. , is an indicator equaling one if stu- 
dent i, in grade g, and school s, was observed to exit 
his school in year t+ 1 ; charter indicates that the stu- 
dent’s school in year t was a charter. X is a vector of 
observed student characteristics; belowtest indicates 
that the students test score is below the average 
score, relative to his school or NY state (depending 
upon the analysis). A and 8 are year and grade fixed 
effects; e is a stochastic term clustered by school in 
year t. And fi represent parameters to be estimated. 

Figure 2 reveals results from estimating Equation 
1 , with each column reporting results of one of six 
individual regressions. The models use other con- 
trols; but for clarity, Figure 2 reports only coefficient 
estimates and standard errors for three variables of 
particular interest. 

Row 1 estimates the independent relationship be- 
tween attending a charter and attrition probability, 


abstracted from students’ observed test-score perfor- 
mance and other factors. In all cases, the estimate is 
negative and statistically significant. Row 1 shows 
that — on average and all else equal — students are 
less likely to exit charters than traditional public 
schools by about 2 percentage points. (This result 
is consistent with recent analyses, cited previously, 
finding that attrition is lower, overall, in NYC char- 
ters than in traditional public schools.) 

Row 2 reports the estimated, independent relation- 
ship between being a low-performing student (as 
defined by the model) on the probability of exit 
in a given year, abstracted from students’ test-score 
performance and other factors. Row 2, like Figure 
1 , reveals that low-performing students are, overall, 
more likely to exit their schools, whether charter or 
traditional public. 

Of particular interest is the coefficient estimate, 
“Interaction: Low-Performing & Charter School.” 
These cells represent the estimated differential im- 
pact — of being a low-performing student in a char- 
ter, relative to being a low-performing student in a 
traditional public school — on the probability of exit 
in a given year. In other words, the coefficient mea- 
sures whether, and by how much, low-performing 
students are more, or less, likely to exit charters rela- 
tive to traditional public schools. 

In all cases, results for the interaction between at- 
tending a charter and being a low-performing stu- 
dent are statistically insignificant at conventional 
levels. Therefore, the estimates find that low-per- 
forming students are statistically just as likely to 
exit a traditional public school as they are to exit a 
charter. 8 (The lone exception: at the 1 0 percent sig- 
nificance level, students who score below the school 
mean on the state English Language Arts exam are 
more likely to exit charters by about 1.93 percent- 
age points.) 

CONCLUSION 

Charter critics correctly observe that low-perform- 
ing students are more likely to exit charters than 
higher-performing students. However, as this pa- 
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Figure 2. Relationship Between Low Performance, Charter Schooling, 

and Probability of Exit 
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[0.0117] 
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Low-Performing in Previous Year 
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[0.00402] 

[0.00254] 
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[0.00325] 

[0.00445] 
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0.0140 

0.0200 
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0.0193* 
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0.0277 
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1,408,693 

1,408,693 

R-Squared 

0.025 

0.026 

0.025 

0.025 

0.025 

0.025 


* statistically significant at p < 0.1 0. ** statistically significant at p < 0.05. *** statistically significant at p < 0.01 . 
Source: Author's calculations, using data set provided by NYC Department of Education 


per’s analysis of enrollment statistics reveals, this 
isolated fact, when asserted tirelessly and without 
proper context, can be highly misleading. 

Low-performing students, it is also true, are typical- 
ly more mobile than their higher-performing peers, 
regardless of the type of school in which they are 
enrolled: low-performing students are more likely to 
exit traditional public schools than high-performing 
students, too. In short, this paper detects no statis- 
tically discernible difference — whether measured 
by basic descriptive statistics or a simple statistical 
model — in the probability that low-performing 
students are more likely to exit charters than tradi- 
tional public schools. 

Thus, if NYC charters’ attrition rates for low-per- 
forming students, relative to high-performing stu- 
dents, are cause for concern, they are equally wor- 
risome for NYC’s traditional public schools. This 
paper’s findings are consistent with two other recent 
reports by the author, which likewise find that ac- 
tual student enrollment patterns are inconsistent 
with the argument that NYC charters systematically 
push out difficult-to-educate students . 9 Using simi- 
lar DOE data, those two papers reveal, respectively, 
that students with disabilities and English Language 
Learners are less likely to exit NYC charters than 
traditional public schools. 


Alas, no statistical analysis can confirm alleged iso- 
lated instances of charters inappropriately removing 
low-performing students. Nonetheless, the author’s 
analyses suggest that rampant claims that NYC 
charters systematically remove low-performing 
students — whether directly or indirectly, through 
“counseling out” — are, at the very least, overstated. 

As a practical matter, understanding the impact of 
NYC charters on student academic performance 
does not require understanding the type of stu- 
dent who exits charters. Extensive empirical re- 
search, using random assignment 10 and propensity 
score-matching 11 techniques, consistently finds that 
students of all types benefit from attending NYC 
charters over traditional public schools. In no way 
do claims that charters’ aggregate test scores are 
artificially inflated by attrition patterns challenge 
such results. (Because charters are public schools, it 
would, of course, be of great concern if they were 
systematically removing low-performing students.) 

For those who continue to assert that charters sys- 
tematically remove low-performing students, the 
burden is now squarely on their shoulders to pro- 
vide empirical evidence to support their claim. As 
the facts stand, their argument is plainly contradict- 
ed by a basic analysis of NYC longitudinal student 
enrollment and test-score data. 
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